Papers with joint training
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering (2023.tacl-1)
Copied to clipboard
Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, Suranga Nanayakkara
| Challenge: | Retrieval Augment Generation (RAG) has been developed for use in open-domain question answering (ODQA) but it is not optimized for use with other specialized domains such as healthcare and news. |
| Approach: | They propose an extension to RAG that can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. |
| Outcome: | The proposed extension can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. |
Understanding language-elicited EEG data by predicting it from a fine-tuned language model (N19-1)
Copied to clipboard
| Challenge: | Existing studies have only found two of the ERPs to be predictable from embeddings of a stream of language. |
| Approach: | They propose to fine tune a language model to predict ERPs by embedding a stream of language into a model that allows them to be more accurate. |
| Outcome: | The proposed model fine tunes the ERPs to predict them for the first time. |
Language Scaling for Universal Suggested Replies Model (2021.naacl-industry)
Copied to clipboard
Qianlan Ying, Payal Bajaj, Budhaditya Deb, Yu Yang, Wei Wang, Bojia Lin, Milad Shokouhi, Xia Song, Yang Yang, Daxin Jiang
| Challenge: | We consider scaling automated suggested replies (SR) to multiple languages for a commercial email application. |
| Approach: | They propose a multi-lingual multi-task continual learning framework with auxiliary tasks and language adapters to train universal language representation across regions. |
| Outcome: | The proposed model reduces catastrophic forgetting and improves cross-lingual transfer across languages while reducing training costs. |
Exploring Interpretability in Event Extraction: Multitask Learning of a Neural Event Classifier and an Explanation Decoder (2020.acl-srw)
Copied to clipboard
| Challenge: | EE is a key requirement for machine learning in many domains, e.g., legal, medical, finance. |
| Approach: | They propose an interpretable approach for event extraction that jointly trains a classifier and a rule decoder for event processing. |
| Outcome: | The proposed approach can be used for semi-supervised learning and its performance improves when trained on automatically-labeled data generated by a rule-based system. |
Hate Speech and Offensive Language Detection in Bengali (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing research on hate speech detection in English does not cover low-resource languages like Bengali. |
| Approach: | They develop an annotated dataset of 10K Bengali posts consisting of 5K actual and 5K Romanized Bengali tweets. |
| Outcome: | The proposed model outperforms other models on training actual and romanized datasets by interpreting the semantic expressions better. |
Multi-Task Dense Retrieval via Model Uncertainty Fusion for Open-Domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multitask dense retrieval are not effective due to corpus inconsistency. |
| Approach: | They propose to train individual dense passage retrievers for different open-domain question-answering tasks and aggregate their predictions during test time. |
| Outcome: | The proposed method achieves state-of-the-art performance on 5 benchmark QA datasets, with up to 10% improvement in top-100 accuracy compared to a joint-training multi-task DPR on SQuAD. |
A Joint Model for Document Segmentation and Segment Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to text segmentation focus on document segmentation and segment labeling separately. |
| Approach: | They propose a method for jointly segmenting a document and labeling segments . they show that S-LSTM reduces segmentation error by 30% on average . |
| Outcome: | The proposed method reduces segmentation error by 30% while improving segment labeling. |
ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems (N19-1)
Copied to clipboard
| Challenge: | Existing methods to regularize neural machine translation are limited in low-resource settings. |
| Approach: | They propose a method that uses regressing word embeddings to regularize neural machine translation. |
| Outcome: | The proposed system improves on a strong baseline and a state-of-the-art system. |
Bootstrapping a Crosslingual Semantic Parser (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in semantic parsing are limited to English but professional translation can be prohibitively expensive. |
| Approach: | They adapt a semantic parser trained on a single language to new languages and multiple domains with minimal annotation. |
| Outcome: | The proposed approach achieves parsing accuracy within 2% of translation using only 50% of training data. |
Quantifying Context Overlap for Training Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach in natural language processing because of its commonalities and differences. |
| Approach: | They propose to summarize recent advances in multi-task learning methods based on their task relatedness into two general multi-step training methods. |
| Outcome: | The proposed methods summarize the tasks and discuss future directions. |
Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to retrieve knowledge-intensive conversations are based on external resources such as Wikipedia databases or search engine results. |
| Approach: | They propose an unsupervised query enhanced approach for knowledge-intensive conversations . they conduct experiments on three knowledge- intensive conversation datasets . |
| Outcome: | The proposed approach performs better than all unsupervised methods across three datasets and achieves competitive performance compared to supervised methods. |
ConTextING: Granting Document-Wise Contextual Embeddings to Graph Neural Networks for Inductive Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Graph neural networks (GNNs) are used to learn document representation from graph structures. |
| Approach: | They propose a unified model with a joint training mechanism to learn from document embeddings and contextual word interactions simultaneously. |
| Outcome: | The proposed model outperforms pure inductive GNNs and BERT-style models . the proposed model also has a joint training mechanism to learn from document embeddings and contextual word interactions simultaneously. |
Quality Estimation-Assisted Automatic Post-Editing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing APE and QE combination strategies have not shown significant performance gains in the field of automatic post-editing (APE). |
| Approach: | They propose to train a model on APE and QE tasks to improve the APE performance by using a multi-task learning methodology that treats both tasks as a 'bargaining game' they also investigate various existing combination strategies and show that their approach achieves state-of-the-art performance for a ‘distant’ language pair, viz., English-Marathi. |
| Outcome: | The proposed model improves on two different language pairs, viz., English-Marathi and English-German. |
BACO: A Background Knowledge- and Content-Based Framework for Citing Sentence Generation (2021.acl-long)
Copied to clipboard
| Challenge: | citing sentences capture salient information in cited papers and the connection between citing and citing papers. |
| Approach: | They propose a BAckground knowledge- and COntent-based framework for citing sentence generation that integrates two types of information: background knowledge and content. |
| Outcome: | The proposed framework outperforms baselines in the citation sentence generation task. |
Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | End-to-end speech translation models can be trained to leverage source text . however, since the input modalities are different, it is difficult to leverage the source text successfully. |
| Approach: | They propose to leverage source transcriptions via pre-training and joint training with ASR and NMT tasks. |
| Outcome: | The proposed model predicts paraphrased transcriptions as an auxiliary task with a single decoder. |
Domain Adaptation for Question Answering via Question Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Question answering systems often experience performance deterioration upon user-generated questions. |
| Approach: | They propose a question classification framework to help QA domains adapt to different domains. |
| Outcome: | The proposed framework improves on state-of-the-art datasets against multiple datasets. |
Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Various advanced neural models have been proposed for reading comprehension, but most models ignore its relations with other answer candidates. |
| Approach: | They propose to model reading comprehension as an extract-then-select two-stage procedure . they first extract answer candidates from passages, then select the final answer by combining information from all candidates. |
| Outcome: | The proposed approach improves state-of-the-art performance on open-domain reading comprehension datasets. |
Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation (2023.findings-emnlp)
Copied to clipboard
Gunsoo Han, Daejin Jo, Daniel Nam, Eunseop Yoon, Taehwan Kwon, Seungeun Rho, Kyoung-Woon On, Chang Yoo, Sungwoong Kim
| Challenge: | Existing knowledge-grounded dialogue generation algorithms require annotated knowledge to generate a response grounded on the retrieved knowledge. |
| Approach: | They propose an efficient algorithm for latent variable modeling that leverages large amount of dialogue data. |
| Outcome: | The proposed algorithm outperforms the supervised learning algorithm on knowledge-grounded dialogue datasets while maintaining efficiency and scalability. |
Continuous Speech Tokenizer in Text To Speech (2025.findings-naacl)
Copied to clipboard
| Challenge: | Autoregressive modeling is a common method for processing language sequences and is effective in token prediction. |
| Approach: | They propose a text-to-speech model based on continuous speech tokens and a continuous tokenizer for speech compression. |
| Outcome: | The proposed model has better continuity and higher estimated Mean Opinion Scores (MoS) this is attributed to better information preservation rate across low and high frequencies in the frequency domain. |
Combining Distant and Direct Supervision for Neural Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Existing methods to train relation extraction with distant supervision use noisy labels and implicitly assumes that all the KB facts are mentioned in the text. |
| Approach: | They propose to combine distant supervision data with additional directly-supervised data to train relation extraction models by using sigmoidal attention weights with max pooling. |
| Outcome: | The proposed method achieves state-of-the-art on the widely used FB-NYT dataset. |
UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot dialogue state tracking (DST) ignore unlabelled data in the target domain. |
| Approach: | They propose to transform zero-shot dialogue state tracking into few-shot DST by utilising unlabelled data via joint and self-training methods. |
| Outcome: | The proposed method improves joint goal accuracy by 8% on general language models in zero-shot scenarios, and can be used in many domains. |
Generate & Rank: A Multi-task Framework for Math Word Problems (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies formalize MWP as a generation task but mathematical expressions are prone to minor mistakes. |
| Approach: | They propose a ranking task for math word problem (MWP) that learns from its own mistakes and distinguishes between correct and incorrect expressions. |
| Outcome: | The proposed model outperforms baselines on the classical Math23k dataset and is 7% higher than the state-of-the-art. |
Interpretable and Compositional Relation Learning by Joint Training with an Autoencoder (P18-1)
Copied to clipboard
| Challenge: | Embedding models for entities and relations are useful for recovering missing facts in knowledge bases. |
| Approach: | They propose a dimension reduction technique by training relations jointly with an autoencoder to capture compositional constraints. |
| Outcome: | The proposed model improves on Knowledge Base Completion tasks with a significantly higher mean rank and better compositional training. |
Learning to Mediate Disparities Towards Pragmatic Communication (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work explores pragmatic reasoning based on Rational Speech Act (RSA) and Theory of Mind in communication (Zhu et al., 2021). |
| Approach: | They propose a framework where the speaker attempts to learn the speaker-listener disparity and adjust the speech accordingly by adding a light-weighted disparity adjustment layer into working memory on top of speaker’s long-term memory system. |
| Outcome: | The proposed framework can learn and adapt to different types of listeners by adding a light-weighted disparity adjustment layer into working memory on top of speaker’s long-term memory system. |
Incorporating Precedents for Legal Judgement Prediction on European Court of Human Rights Cases (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Inspired by the legal doctrine of stare decisis, we explore methods to integrate precedents into LJP models. |
| Approach: | They propose to integrate precedents into legal judgment prediction models by integrating them at inference and during training via a precedent fusion module. |
| Outcome: | The proposed model outperforms models without precedents or with precedents incorporated only at inference on LJP tasks. |
A Multi-Level Attention Model for Evidence-Based Fact Checking (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent state-of-the-art approaches have developed increasingly sophisticated models based on graph structures. |
| Approach: | They propose a simple model that can be trained on sequence structures and can benefit from joint training. |
| Outcome: | The proposed model outperforms the graph-based models on a large-scale dataset for Fact Extraction and VERification. |
Counter-Interference Adapter for Multilingual Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multilingual machine translation suffer from performance degradation, resulting in a single model being inferior to separately trained bilingual models on resource-rich languages. |
| Approach: | They propose a transformer-based model with a small parameter overhead for multilingual machine translation that outperforms strong multilingual baselines on 64 of 66 language directions. |
| Outcome: | The proposed model outperforms strong multilingual baselines on 64 of 66 language directions, 42 of which have above 0.5 BLEU improvement. |
Dynamic Fisher-weighted Model Merging via Bayesian Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing merging approaches involve scaling the parameters model-wise or integrating parameter importance parameter-wise. |
| Approach: | They propose a method for merging model-based models at the parameter level without training data or joint training. |
| Outcome: | The proposed model merging framework outperforms baseline models on validation sets. |
On Sparsifying Encoder Outputs in Sequence-to-Sequence Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Using sequence-to-sequence models, encoder outputs are usually transferred to the decoder for generation, but in this study, encoded outputs can be compressed to shorten the sequence for decoding. |
| Approach: | They propose to use a stochastic gate-based algorithm to mask encoder outputs to shorten the sequence delivered for decoding. |
| Outcome: | The proposed model can be used to shorten encoder outputs to short a sequence . the proposed model yields a speedup of up to 1.65 on document summarization and 1.20 on character-based machine translation tasks. |
TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods focus on a single type of distortion and struggle to generalize to unseen scenarios. |
| Approach: | They propose a vision-language model that combines a question-aware visual amplifier module with a large-scale instruction dataset to support training. |
| Outcome: | The proposed model is able to generalize to multiple distortion types while requiring task-specific skills. |
All-in-one: Multi-task Learning for Rumour Verification (C18-1)
Copied to clipboard
| Challenge: | Automatic resolution of rumours is a challenging task that can be broken down into smaller components that make up a pipeline . previous work focused on rumor detection, rumou tracking and stance classification as separate components . |
| Approach: | They propose a multi-task learning approach that allows joint training of main and auxiliary tasks, improving the performance of rumour verification. |
| Outcome: | The proposed approach improves the performance of rumour verification by combining main and auxiliary tasks into one pipeline. |
Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, the generalized intent classification system only considers one stage of OOD learning and requires all IND data for joint training. |
| Approach: | They propose a task that detects OOD intents from dynamic OOD data streams . they propose CGID method that bootstraps new intent discovery through class prototypes . |
| Outcome: | The proposed task can detect out-of-domain (OOD) queries and extend them to the in-domain classifier . it can safely and efficiently detect out of-domain queries and avoid wrong operations . |
Does Joint Training Really Help Cascaded Speech Translation? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, in speech translation, the straightforward approach delivers state-of-the-art results, but fundamental challenges such as error propagation remain. |
| Approach: | They propose to combine a cascaded recognition system with a machine translation system to improve cascade speech translation. |
| Outcome: | The proposed methods can improve cascaded speech translation and suggest alternative training methods. |
Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets (D18-1)
Copied to clipboard
| Challenge: | Existing large labeled text datasets contain labels for multiple subsets of biomedical entity types, but it is rare to find large labeling datasets containing all desired entity types together. |
| Approach: | They propose a method for training a single CRF extractor from multiple datasets with disjoint or partially overlapping sets of entity types. |
| Outcome: | The proposed method improves NER F1 over training in isolation on biocreative V CDR, biocreativ VI ChemProt and MedMentions datasets. |
Empathy Intent Drives Empathy Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Empathy is essential in human social interaction. |
| Approach: | They propose to annotate healthy empathy detection datasets IEMPATHIZE and TwittEmp with 8 empathy intent labels and perform joint training for the two tasks. |
| Outcome: | The proposed framework outperforms baselines on the two datasets. |
Lemmatisation & Morphological Analysis of Unedited Greek: Do Simple Tasks Need Complex Solutions? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for part-of-speech tagging of unedited Greek text have outperformed traditional methods for morphological analysis and lemmatisation. |
| Approach: | They propose to combine nominal features into a single label and combine the three most distinctive features of verbs into another unified label. |
| Outcome: | The proposed models outperform traditional models in lemmatisation and morphological analysis and show that multi-task learning improves performance by transferring parameters. |
On the Cross-lingual Transferability of Monolingual Representations (2020.acl-main)
Copied to clipboard
| Challenge: | State-of-the-art unsupervised multilingual models generalize in zero-shot cross-lingual setting . generalization ability attributed to shared subword vocabulary and joint training across multiple languages . |
| Approach: | They propose an approach that transfers a monolingual model to new languages at the lexical level. |
| Outcome: | The proposed approach is competitive with multilingual BERT on cross-lingual classification benchmarks and on a new cross-linguistic question answering dataset. |
UniMath: A Foundational and Multimodal Mathematical Reasoner (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for interpreting and processing diverse mathematical modalities are limited . existing systems are limited in interpreting complex mathematical tasks and implementing them in a multimodal manner. |
| Approach: | They propose a multimodal mathematical reasoning system that utilizes a fine-tuned T5 model augmented with a variational autoencoder (VAE)-based image tokenizer. |
| Outcome: | The proposed model achieves state-of-the-art performance on SVAMP, GeoQA, and TableMWP datasets and is generalized on two additional datasets. |
Unifying the Convergences in Multilingual Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation are overfitting and inconsistency is ignored . |
| Approach: | They propose a training strategy that picks up language-specific best checkpoints for each language pair to teach the current model on the fly. |
| Outcome: | The proposed training strategy alleviates convergence inconsistency and achieves state-of-the-art on language pairs. |
Consecutive Question Generation via Dynamic Multitask Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | , . ; ) ()((); ()) .())((2): ""(). |
| Approach: | They propose a sequential sequential question-answer generation task with auxiliary tasks generating other elements to generate logically related question-anchor pairs to understand a passage. |
| Outcome: | The proposed framework improves question generation significantly and benefit multiple related tasks. |
Cross-Register Projection for Headline Part of Speech Tagging (2021.emnlp-main)
Copied to clipboard
| Challenge: | Part of speech (POS) tagging models are underperforming on headlines due to differences in the register of English news headlines and long-form text. |
| Approach: | They propose to annotate news headlines with POS tags by projecting predicted tags from corresponding sentences in news bodies. |
| Outcome: | The proposed model reduces errors by 23% and 19% on a newly-annotated corpus of over 5,248 English news headlines from the Google sentence compression corpus. |
Reward Modeling for Scientific Writing Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for scientific writing evaluation are primarily optimized for general-purpose benchmarks with fixed scoring rubrics and evaluation criteria. |
| Approach: | They propose to train scientific writing evaluation models that leverage domain knowledge . they use a two-stage evaluation framework that optimizes evaluation preferences and refines reasoning capabilities . |
| Outcome: | The proposed model generalizes effectively across tasks and to previously unseen settings. |
Model Composition for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Chi Chen, Yiyang Du, Zheng Fang, Ziyue Wang, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu
| Challenge: | Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities. |
| Approach: | They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters. |
| Outcome: | The proposed model retains the modal understanding capabilities of each original model. |
A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis (2024.acl-long)
Copied to clipboard
| Challenge: | Existing work on aspect-based sentiment analysis (ABSA) focuses on sentence level, document level ABSA is more practical and requires holistic document-level understanding capabilities. |
| Approach: | They propose a learning framework to jointly model the DTSA task and the coreference resolution task using ChatGPT. |
| Outcome: | The proposed framework reduces the reliance on annotated coreference information and alleviates evaluation bias caused by missing coreference targets. |
GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domains (2024.emnlp-main)
Copied to clipboard
Yang Janet Liu, Tatsuya Aoyama, Wesley Scivetti, Yilun Zhu, Shabnam Behzad, Lauren Levine, Jessica Lin, Devika Tiwari, Amir Zeldes
| Challenge: | Existing shallow discourse parsing systems focus on the Wall Street Journal corpus, but the data is limited to the news domain and is 35 years old. |
| Approach: | They propose to use the Wall Street Journal corpus as a benchmark for PDTB-style shallow discourse parsing. |
| Outcome: | The proposed dataset is compatible with PDTB, but suffers from degradation out-of-domain. |
Large-scale Lifelong Learning of In-context Instructions and How to Tackle It (2023.acl-long)
Copied to clipboard
| Challenge: | In-context instruction learning is a method to improve the target PLM’s instance- and task-level generalization performance as it observes more tasks. |
| Approach: | They propose to fine-tune a Pre-trained Language Model (PLM) on a set of tasks with in-context instructions and to extend this property to a scenario in which tasks are fed to the target PLM in a sequential manner. |
| Outcome: | The proposed method achieves noticeable improvements in both types of generalization, nearly reaching the upper bound performance obtained through joint training. |
Learning Opinion Summarizers by Selecting Informative Reviews (2021.emnlp-main)
Copied to clipboard
| Challenge: | supervised summarization has been traditionally approached with unsupervised, weakly-supervised and few-shot learning techniques. |
| Approach: | They propose to combine a large dataset of opinion summaries with user reviews to form a supervised summarizer. |
| Outcome: | The proposed method improves the quality of summarization and reduces hallucinations in the summarizer. |
TEPrompt: Task Enlightenment Prompt Learning for Implicit Discourse Relation Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt learning models for IDRR use multiple-prompt decisions from three different yet much similar connective prediction templates. |
| Approach: | They propose to fuse three related tasks to fuse the learned features of auxiliary tasks to create a prompt learning model that can be used to boost the main task. |
| Outcome: | The proposed model outperforms the ConnPrompt in the training phase and in the testing phase. |
Transforming Sequence Tagging Into A Seq2Seq Task (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained, large, generative language models have had great success in a wide range of sequence tagging and structured prediction tasks. |
| Approach: | They propose to use a new format for casting input text sentences and their output labels into the input and target of a Seq2Seq model and introduce it to test their hypothesis. |
| Outcome: | The proposed format shows to be both simpler and more effective and devoid of hallucination. |
Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization (2025.acl-long)
Copied to clipboard
Lei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan, Xiachong Feng, Yangfan Ye, Weihong Zhong, Yuxuan Gu, Baoxin Wang, Dayong Wu, Guoping Hu, Bing Qin
| Challenge: | Existing frameworks for retrieval-augmented large language models (LLMs) are lacking in LFQA faithfulness testing. |
| Approach: | They propose a framework to teach retrieval-augmented large language models to explicitly discriminate between faithful and unfaithful generations. |
| Outcome: | The proposed framework outperforms GPT-4o in LFQA scenarios and outperformed existing benchmarks. |
Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | a new study addresses bias and stereotypes in language models by exploring how learning them together improves performance. |
| Approach: | They propose a dataset for bias and stereotype detection that integrates religion, gender, socio-economic status, race, profession, and others. |
| Outcome: | The proposed dataset compares encoder-only models and fine-tuned decoder- only models . the results show that learning stereotypes together improves bias detection . |
DEM: Distribution Edited Model for Training with Mixed Data Distributions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent fine-tuning approaches for large language models require supervised finetun on diverse datasets and follow different distributions. |
| Approach: | They propose a distribution edited model that integrates models individually trained on each data source with the base model using basic element-wise vector operations. |
| Outcome: | The proposed model outperforms baseline models on a variety of benchmarks and is cheaper than standard data mixing methods. |
Logos as a Well-Tempered Pre-train for Sign Language Recognition (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing data on sign languages is limited, resulting in ambiguity in dataset labeling . similar signs can have different semantic meanings, which leads to ambiguous labeling. |
| Approach: | They propose to use a Russian sign language dataset as a universal encoder for other languages . they propose to explicitly annotate visually similar signs to improve model quality . |
| Outcome: | The proposed model outperforms current state-of-the-art models and gets competitive results for the AUTSL dataset. |
UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions (2026.acl-long)
Copied to clipboard
Chunyu Qiang, Xiaopeng Wang, Kang Yin, Yuzhe Liang, Yuxin Guo, Teng Ma, Ziyu Zhang, Tianrui Wang, Cheng Gong, Yushen Chen, Ruibo Fu, Longbiao Wang, Jianwu Dang
| Challenge: | Generative audio modeling has been fragmented into specialized tasks such as text-to-speech (TTS), text- to-music (TTM), and text-ta (TTA) specialized models require reference audio for timbre cloning and strict phoneme alignment, whereas TTA models generate unstructured textures from open-ended captions. |
| Approach: | They propose a unified flow-matching framework capable of synthesizing speech, music, sound effects . they propose 'token injection mechanism' that projects unstructured environmental sounds into structured temporal latent space . |
| Outcome: | The proposed framework achieves state-of-the-art performance in instruction-based TTS and TTM while maintaining competitive fidelity in TTA. |
xCoRe: Cross-context Coreference Resolution (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current coreference resolution systems are limited to short-to-medium-sized documents and struggle to scale to very long documents due to architectural limitations and implied memory costs. |
| Approach: | They propose a unified approach to coreference resolution that unifies two challenging settings . they use a pipeline that first identifies mentions, then creates clusters within individual contexts . |
| Outcome: | The proposed model achieves state-of-the-art results on cross-document benchmarks and strong performance on long-document data while retaining top-tier results on traditional datasets. |